\section {The Language-Independent Part}
\label {lipp}
@ The set of characters that may begin an interesting token.
<<Definition of interesting tokens>>=
<<Csets for comment, quote, and special tokens>>
begin_token := res_word_chars ++ begin_comment1 ++ begin_comment2 ++ begin_quote ++
	       begin_special
@ These sets contain the {\it initial\/} characters of the corresponding sets of tokens
(strings). The token recognition method used in procedure [[TeXify]] is based on the
assumption that these sets are mutually disjoint, and that they also do not intersect the
set [[id_chars]]. If this assumption does not hold, the results are unpredictable.
<<Csets for comment, quote, and special tokens>>=
begin_comment1 := begin_comment2 := begin_quote := begin_special := ''
every e := !comment1 do begin_comment1 ++:= cset(e[1])
every e := !comment2 do begin_comment2 ++:= cset(e[1][1])
every e := !quote do begin_quote ++:= cset(e[1])
every e := !special do begin_special ++:= cset(e[1])
<<Check that these sets are disjoint>>
\subsection {The procedure TeXify}
Icon's braces for grouping are horrible! Of course, we owe all of this (and more) to C.
<<Procedure [[TeXify]]>>=
procedure TeXify (line)
  <<Local variables>>
  writes("@literal ")
  line ? {if /in_comment then 
	    while writes(tab(upto(begin_token))) do
	      case &pos+1 of {
		any(id_chars) : <<Identifier>>
		any(begin_special) : <<Possible ``special'' token>>
		any(begin_comment1) : <<Unbalanced comment token>>
		any(begin_comment2) : <<Balanced comment token>>
		any(begin_quote) : <<Quoting token>>
		default: stop("\n** Error in nw.TeXify: pos = ", &pos)
	      }
	  else
	    <<Write comment>>
	  writes(tab(0))
  write()
  return
@ All identifiers will be matched (and some non-identifiers, such as explicit numeric
constants), but translations are defined only for reserved words.
<<Identifier>>=
{token := tab(many(id_chars))
 <<Write the translation or the token itself>>}
@ Since the set [[special]] may contain strings which are prefixes of other strings in
[[special]], we can't check for a special token just by [[tab(match(!special))]]. That's
why the set [[S]] is needed.
<<Possible ``special'' token>>=
{token := tab(many(S))
 <<Write the translation or the token itself>>}
@ Output the token and the rest of the line as is, but in [[\rm]] font. Within a comment,
characters special to \TeX\ are active, e.g. \verb+$x^2$+ will produce $x^2$.
<<Unbalanced comment token>>=
{writes(tab(match(!comment1)) || "\\begcom" || tab(0) || "\\endcom")
 break}
@ If we are here, it is not necessarily true that we have found a comment. For example, in
{\sl Mathematica\/} comments begin with a [[(]], which may also appear in [[x+(y+z)]].
<<Balanced comment token>>=
{every c := !comment2 do
   {writes(c_open := tab(match(c[1]))) & c_close := c[2] & break} # Nice Icon style!
 if \c_open then
   {in_comment := "yes"
    writes("\\begcom")
    <<Write comment>>}
 else
   writes(move(1)) # The character wasn't the beginning of a comment.
@ After encountering a quote we write literally, except that we precede every character
special to \TeX\ by a backslash and follow it by an empty group. (This is necessary for
the characters ``\~{}'' and ``\^{}''.)
<<Quoting token>>=
{writes(q := tab(match(!quote))) # The opening quote.
 Q := tab(upto(q))  # The closing quote has to be of the same kind.
 Q ? {while writes(tab(upto(TeXspecial))) do
	writes("\\" || move(1) || "{}")
      writes(tab(0))
     }
 writes(tab(match(!quote))) # The closing quote, hopefully q.
<<Write the translation or the token itself>>=
t := translation[\token]
writes(if \t then t else token)
@ We write literally, but in [[\rm]] font. Within a comment, characters special to \TeX\
are active, e.g. \verb+$x^2$+ will produce $x^2$.
<<Write comment>>=
{if writes(tab(find(c_close))) then
   # Comment ends here.
   {writes("\\endcom" || move(*c_close))
    in_comment := &null}
 else
   # Comment doesn't close on this line.
   writes(tab(0)) 
<<Check that these sets are disjoint>>=
I := begin_comment1 ** begin_comment2 ** begin_quote ** begin_special ** id_chars
if *I ~= 0 then stop ("** nw problem: the characters in the set ", image(I),
                      "\n may begin tokens in more than one category!")
@ [[in_comment]] is [[true]] if we are inside a comment, and [[&null]] otherwise.
[[TeXspecial]] is the set of characters treated specially by \TeX.
<<Local variables>>=
local token, c, t, I, q, Q, c_open
static in_comment, c_close, TeXspecial
initial {TeXspecial := '\\${}&#^_%~'}
<<Global declarations>>=
global translation, id_chars, begin_token, begin_special, S, begin_comment1
global  begin_comment2, begin_quote, special, comment1, comment2, quote
\subsection {Filtering the Input}
@ First we set up the Courier bold font [[\Cb]] (a Postscript font). Then we define the
macro [[\inwcode]] to activate ``[[$]]'' in code sections, and the macro [[\begcom]]
(begin comment) for switching to [[\rm]] font and making the characters ``{\tt\^{}}'', and
``[[_]]'' active within comments. (``[[$]]'' will also be active within comments, since
they occur only in code sections.)
This section has been written mostly by N. Ramsey.
<<Emit special {\TeX} definitions>>=
write("@literal \\font\\Cb=pcrb ")
write("@literal \\def\\inwcode{\\catcode`\\$=3} ")
write("@literal \\def\\begcom{\\begingroup\\rm\\catcode`\\^=7\\catcode`\\_=8{}} ")
write("@literal \\def\\endcom{\\endgroup} ")
<<Read and filter all the input lines>>=
while line := read() do
  line ? (="@" & filter(tab(upto(' ')|0), if =" " then tab(0) else &null))
<<Procedure [[filter]]>>=
procedure filter (name, arg)
  static kind
  case name of {
    "begin" : {arg ? kind := tab(many(&letters)) 
		 copyline(name, arg)
		 if kind == "code" then write("@literal \\inwcode")
	      }
    "text"  : if \kind == "code" then TeXify(arg) else copyline(name, arg)
    default : copyline(name, arg)
  return
procedure copyline(name, arg)
  return write("@", name, (" " || \arg) | "")
\section{Chunks}
\nowebchunks
%\twocolumn[\section{Index}]
\section{Index}
\nowebindex
\end {document}